Individual Contrast Preferences in Natural Images.

This paper is an investigation in the field of personalized image quality assessment with the focus of studying individual contrast preferences for natural images. To achieve this objective, we conducted an in-lab experiment with 22 observers who assessed 499 natural images and collected their contrast level preferences. We used a three-alternative forced choice comparison approach coupled with a modified adaptive staircase algorithm to dynamically adjust the contrast for each new triplet. Through cluster analysis, we clustered observers into three groups based on their preferred contrast ranges: low contrast, natural contrast, and high contrast. This finding demonstrates the existence of individual variations in contrast preferences among observers. To facilitate further research in the field of personalized image quality assessment, we have created a database containing 10,978 original contrast level values preferred by observers, which is publicly available online.

Evaluating clinical diversity and plausibility of synthetic capsule endoscopic images.

Wireless Capsule Endoscopy (WCE) is being increasingly used as an alternative imaging modality for complete and non-invasive screening of the gastrointestinal tract. Although this is advantageous in reducing unnecessary hospital admissions, it also demands that a WCE diagnostic protocol be in place so larger populations can be effectively screened. This calls for training and education protocols attuned specifically to this modality. Like training in other modalities such as traditional endoscopy, CT, MRI, etc., a WCE training protocol would require an atlas comprising of a large corpora of images that show vivid descriptions of pathologies, ideally observed over a period of time. Since such comprehensive atlases are presently lacking in WCE, in this work, we propose a deep learning method for utilizing already available studies across different institutions for the creation of a realistic WCE atlas using StyleGAN. We identify clinically relevant attributes in WCE such that synthetic images can be generated with selected attributes on cue. Beyond this, we also simulate several disease progression scenarios. The generated images are evaluated for realism and plausibility through three subjective online experiments with the participation of eight gastroenterology experts from three geographical locations and a variety of years of experience. The results from the experiments indicate that the images are highly realistic and the disease scenarios plausible. The images comprising the atlas are available publicly for use in training applications as well as supplementing real datasets for deep learning.

Concept-based reasoning in medical imaging.

PURPOSE: As concept-based reasoning for improving model interpretability becomes promising, the question of how to define good concepts becomes more pertinent. In domains like medical, it is not always feasible to access instances clearly representing good concepts. In this work, we propose an approach to use organically mined concepts from unlabeled data to explain classifier predictions. METHODS: A Concept Mapping Module (CMM) is central to this approach. Given a capsule endoscopy image predicted as abnormal, the CMM's main task is to identify which concept explains the abnormality. It consists of two parts, namely a convolutional encoder and a similarity block. The encoder maps the incoming image into the latent vector, while the similarity block retrieves the closest aligning concept as explanation. RESULTS: Abnormal images can be explained in terms of five pathology-related concepts retrieved from the latent space given by inflammation (mild and severe), vascularity, ulcer and polyp. Other non-pathological concepts found include anatomy, debris, intestinal fluid and capsule modality. CONCLUSIONS: This method outlines an approach through which concept-based explanations can be generated. Exploiting the latent space of styleGAN to look for variations and using task-relevant variations for defining concepts is a powerful way through which an initial concept dictionary can be created which can subsequently be iteratively refined with much less time and resource.

SSP-Net: A Siamese-based Structure-Preserving Generative Adversarial Network for Unpaired Medical Image Enhancement.

Recently, unpaired medical image enhancement is one of the important topics in medical research. Although deep learning-based methods have achieved remarkable success in medical image enhancement, such methods face the challenge of low-quality training sets and the lack of a large amount of data for paired training data. In this paper, a dual input mechanism image enhancement method based on Siamese structure (SSP-Net) is proposed, which takes into account the structure of target highlight (texture enhancement) and background balance (consistent background contrast) from unpaired low-quality and high-quality medical images. Furthermore, the proposed method introduces the mechanism of the generative adversarial network to achieve structure-preserving enhancement by jointly iterating adversarial learning. Experiments comprehensively illustrate the performance in unpaired image enhancement of the proposed SSP-Net compared with other state-of-the-art techniques.

Image quality and perception: introduction.

This feature issue focuses on image quality and perception, including image and video quality, subjective and objective quality, and enhancement. The feature issue contains papers on several important topics, such as contrast discrimination, analysis of color imaging in cameras, image quality assessment, and more. The papers represent different important aspects in image quality and perception, contributing to the advancement of the field.

Modification and evaluation of the peripheral contrast sensitivity function models.

We propose a series of modifications to the Barten contrast sensitivity function model for peripheral vision based on anatomical and psychophysical studies. These modifications result in a luminance pattern detection model that could quantitatively describe the extent of veridical pattern resolution and the aliasing zone. We evaluated our model against psychophysical measurements in peripheral vision. Our numerical assessment shows that the modified Barten leads to lower estimate errors than its original version.

From labels to priors in capsule endoscopy: a prior guided approach for improving generalization with few labels.

The lack of generalizability of deep learning approaches for the automated diagnosis of pathologies in Wireless Capsule Endoscopy (WCE) has prevented any significant advantages from trickling down to real clinical practices. As a result, disease management using WCE continues to depend on exhaustive manual investigations by medical experts. This explains its limited use despite several advantages. Prior works have considered using higher quality and quantity of labels as a way of tackling the lack of generalization, however this is hardly scalable considering pathology diversity not to mention that labeling large datasets encumbers the medical staff additionally. We propose using freely available domain knowledge as priors to learn more robust and generalizable representations. We experimentally show that domain priors can benefit representations by acting in proxy of labels, thereby significantly reducing the labeling requirement while still enabling fully unsupervised yet pathology-aware learning. We use the contrastive objective along with prior-guided views during pretraining, where the view choices inspire sensitivity to pathological information. Extensive experiments on three datasets show that our method performs better than (or closes gap with) the state-of-the-art in the domain, establishing a new benchmark in pathology classification and cross-dataset generalization, as well as scaling to unseen pathology categories.

Effect of Elevation and Surface Roughness on Naturalness Perception of 2.5D Decor Prints.

Naturalness is a complex concept. It can involve a variety of attributes. In this work, we considered the effect of elevation and surface roughness on naturalness perception of 2.5D decor prints for four material categories. We found that elevation has an impact on the naturalness perception of 2.5D decor prints and that it is linked with content. The observers found lower elevation to be more natural for wood and glass 2.5D prints while there was no clear tendency for stone and metal 2.5D prints. We also found the perceptual attributes used for naturalness assessment of 2.5D decor prints. The top five ones are color, roughness, gloss, elevation, and lightness. The obtained findings can be useful for companies that produce 2.5D prints.

What Elevation Makes 2.5D Prints Perceptually Natural?

Elevation plays a considerable role in naturalness perception of 2.5D prints. The necessary level of elevation to make 2.5D prints look perceptually natural may vary from application to application. Therefore, one needs to know the right elevation for specific applications to make the prints look perceptually natural. In this work, we investigated what elevation makes 2.5D prints of wood images perceptually natural. We worked with various wood content images such as wooden wicker, wall, roof, and floor. We found that the optimal elevation that makes 2.5D prints of wood images perceptually natural is content-dependent and in a range between 0.3 mm and 0.5 mm. Moreover, we found that the optimal elevation becomes 0.5 mm if we consider images of wood regardless of the wood content. In addition, there was a high correlation between majority of observers on naturalness perception of 2.5D prints of wood images.

Translucency perception: A review.

Translucency is an optical and a perceptual phenomenon that characterizes subsurface light transport through objects and materials. Translucency as an optical property of a material relates to the radiative transfer inside and through this medium, and translucency as a perceptual phenomenon describes the visual sensation experienced by humans when observing a given material under given conditions. The knowledge about the visual mechanisms of the translucency perception remains limited. Accurate prediction of the appearance of the translucent objects can have a significant commercial impact in the fields such as three-dimensional printing. However, little is known how the optical properties of a material relate to a perception evoked in humans. This article overviews the knowledge status about the visual perception of translucency and highlights the applications of the translucency perception research. Furthermore, this review summarizes current knowledge gaps, fundamental challenges and existing ambiguities with a goal to facilitate translucency perception research in the future.

Three-stage segmentation of lung region from CT images using deep neural networks.

BACKGROUND: Lung region segmentation is an important stage of automated image-based approaches for the diagnosis of respiratory diseases. Manual methods executed by experts are considered the gold standard, but it is time consuming and the accuracy is dependent on radiologists' experience. Automated methods are relatively fast and reproducible with potential to facilitate physician interpretation of images. However, these benefits are possible only after overcoming several challenges. The traditional methods that are formulated as a three-stage segmentation demonstrate promising results on normal CT data but perform poorly in the presence of pathological features and variations in image quality attributes. The implementation of deep learning methods that can demonstrate superior performance over traditional methods is dependent on the quantity, quality, cost and the time it takes to generate training data. Thus, efficient and clinically relevant automated segmentation method is desired for the diagnosis of respiratory diseases. METHODS: We implement each of the three stages of traditional methods using deep learning methods trained on five different configurations of training data with ground truths obtained from the 3D Image Reconstruction for Comparison of Algorithm Database (3DIRCAD) and the Interstitial Lung Diseases (ILD) database. The data was augmented with the Lung Image Database Consortium (LIDC-IDRI) image collection and a realistic phantom. A convolutional neural network (CNN) at the preprocessing stage classifies the input into lung and none lung regions. The processing stage was implemented using a CNN-based U-net while the postprocessing stage utilize another U-net and CNN for contour refinement and filtering out false positives, respectively. RESULTS: The performance of the proposed method was evaluated on 1230 and 1100 CT slices from the 3DIRCAD and ILD databases. We investigate the performance of the proposed method on five configurations of training data and three configurations of the segmentation system; three-stage segmentation and three-stage segmentation without a CNN classifier and contrast enhancement, respectively. The Dice-score recorded by the proposed method range from 0.76 to 0.95. CONCLUSION: The clinical relevance and segmentation accuracy of deep learning models can improve though deep learning-based three-stage segmentation, image quality evaluation and enhancement as well as augmenting the training data with large volume of cheap and quality training data. We propose a new and novel deep learning-based method of contour refinement.

Blind Quality Assessment of Iris Images Acquired in Visible Light for Biometric Recognition.

Image quality is a key issue affecting the performance of biometric systems. Ensuring the quality of iris images acquired in unconstrained imaging conditions in visible light poses many challenges to iris recognition systems. Poor-quality iris images increase the false rejection rate and decrease the performance of the systems by quality filtering. Methods that can accurately predict iris image quality can improve the efficiency of quality-control protocols in iris recognition systems. We propose a fast blind/no-reference metric for predicting iris image quality. The proposed metric is based on statistical features of the sign and the magnitude of local image intensities. The experiments, conducted with a reference iris recognition system and three datasets of iris images acquired in visible light, showed that the quality of iris images strongly affects the recognition performance and is highly correlated with the iris matching scores. Rejecting poor-quality iris images improved the performance of the iris recognition system. In addition, we analyzed the effect of iris image quality on the accuracy of the iris segmentation module in the iris recognition system.

Weakly-Supervised Network for Detection of COVID-19 in Chest CT Scans.

Deep Learning-based chest Computed Tomography (CT) analysis has been proven to be effective and efficient for COVID-19 diagnosis. Existing deep learning approaches heavily rely on large labeled data sets, which are difficult to acquire in this pandemic situation. Therefore, weakly-supervised approaches are in demand. In this paper, we propose an end-to-end weakly-supervised COVID-19 detection approach, ResNext+, that only requires volume level data labels and can provide slice level prediction. The proposed approach incorporates a lung segmentation mask as well as spatial and channel attention to extract spatial features. Besides, Long Short Term Memory (LSTM) is utilized to acquire the axial dependency of the slices. Moreover, a slice attention module is applied before the final fully connected layer to generate the slice level prediction without additional supervision. An ablation study is conducted to show the efficiency of the attention blocks and the segmentation mask block. Experimental results, obtained from publicly available datasets, show a precision of 81.9% and F1 score of 81.4%. The closest state-of-the-art gives 76.7% precision and 78.8% F1 score. The 5% improvement in precision and 3% in the F1 score demonstrate the effectiveness of the proposed method. It is worth noticing that, applying image enhancement approaches do not improve the performance of the proposed method, sometimes even harm the scores, although the enhanced images have better perceptual quality.

A Three-Feature Model to Predict Colour Change Blindness.

Change blindness is a striking shortcoming of our visual system which is exploited in the popular `Spot the difference' game, as it makes us unable to notice large visual changes happening right before our eyes. Change blindness illustrates the fact that we see much less than we think we do. In this paper, we introduce a fully automated model to predict colour change blindness in cartoon images based on image complexity, change magnitude and observer experience. Using linear regression with only three parameters, the predictions of the proposed model correlate significantly with measured detection times. We also demonstrate the efficacy of the model to classify stimuli in terms of difficulty.

Local Indicators of Spatial Autocorrelation (LISA): Application to Blind Noise-Based Perceptual Quality Metric Index for Magnetic Resonance Images.

Noise-based quality evaluation of MRI images is highly desired in noise-dominant environments. Current noise-based MRI quality evaluation methods have drawbacks which limit their effective performance. Traditional full-reference methods such as SNR and most of the model-based techniques cannot provide perceptual quality metrics required for accurate diagnosis, treatment and monitoring of diseases. Although techniques based on the Moran coefficients are perceptual quality metrics, they are full-reference methods and will be ineffective in applications where the reference image is not available. Furthermore, the predicted quality scores are difficult to interpret because their quality indices are not standardized. In this paper, we propose a new no-reference perceptual quality evaluation method for grayscale images such as MRI images. Our approach is formulated to mimic how humans perceive an image. It transforms noise level into a standardized perceptual quality score. Global Moran statistics is combined with local indicators of spatial autocorrelation in the form of local Moran statistics. Quality score is predicted from perceptually weighted combination of clustered and random pixels. Performance evaluation, comparative performance evaluation and validation by human observers, shows that the proposed method will be a useful tool in the evaluation of retrospectively acquired MRI images and the evaluation of noise reduction algorithms.

Standardized quality metric system for structural brain magnetic resonance images in multi-center neuroimaging study.

BACKGROUND: Multi-site neuroimaging offer several benefits and poses tough challenges in the drug development process. Although MRI protocol and clinical guidelines developed to address these challenges recommend the use of good quality images, reliable assessment of image quality is hampered by the several shortcomings of existing techniques. METHODS: Given a test image two feature images are extracted. They are grayscale and contrast feature images. Four binary images are generated by setting four different global thresholds on the feature images. Image quality is predicted by measuring the structural similarity between appropriate pairs of binary images. The lower and upper limits of the quality index are 0 and 1. Quality prediction is based on four quality attributes; luminance contrast, texture, texture contrast and lightness. RESULTS: Performance evaluation on test data from three multi-site clinical trials show good objective quality evaluation across MRI sequences, levels of distortion and quality attributes. Correlation with subjective evaluation by human observers is ≥ 0.6. CONCLUSION: The results are promising for the evaluation of MRI protocols, specifically the standardization of quality index, designed to overcome the challenges encountered in multi-site clinical trials.

Image Quality Evaluation in Clinical Research: A Case Study on Brain and Cardiac MRI Images in Multi-Center Clinical Trials.

Magnetic resonance imaging (MRI) system images are important components in the development of drugs because it can reveal the underlying pathology in diseases. Unfortunately, the processes of image acquisition, storage, transmission, processing, and analysis can influence image quality with the risk of compromising the reliability of MRI-based data. Therefore, it is necessary to monitor image quality throughout the different stages of the imaging workflow. This report describes a new approach to evaluate the quality of an MRI slice in multi-center clinical trials. The design philosophy assumes that an MRI slice, such as all natural images, possess statistical properties that can describe different levels of contrast degradation. A unique set of pixel configuration is assigned to each possible level of contrast-distorted MRI slice. Invocation of the central limit theorem results in two separate Gaussian distributions. The central limit theorem says that the mean and standard deviation of pixel configuration assigned to each possible level of contrast degradation will follow a normal distribution. The mean of each normal distribution corresponds to the mean and standard deviation of the underlying ideal image. Quality prediction processes for a test image can be summarized into four steps. The first step extracts local contrast feature image from the test image. The second step computes the mean and standard deviation of the feature image. The third step separately standardizes each normal distribution using the mean and standard deviation computed from the feature image. This gives two separate z-scores. The fourth step predicts the lightness contrast quality score and the texture contrast quality score from cumulative distribution function of the appropriate normal distribution. The proposed method was evaluated objectively on brain and cardiac MRI volume data using four different types and levels of degradation. The four types of degradation are Rician noise, circular blur, motion blur, and intensity nonuniformity also known as bias fields. Objective evaluation was validated using a proposed variation of difference of mean opinion scores. Results from performance evaluation show that the proposed method will be suitable to monitor and standardize image quality throughout the different stages of imaging workflow in large clinical trials. MATLAB implementation of the proposed objective quality evaluation method can be downloaded from (https://github.com/ezimic/Image-Quality-Evaluation).

Blind blur assessment of MRI images using parallel multiscale difference of Gaussian filters.

BACKGROUND: Rician noise, bias fields and blur are the common distortions that degrade MRI images during acquisition. Blur is unique in comparison to Rician noise and bias fields because it can be introduced into an image beyond the acquisition stage such as postacquisition processing and the manifestation of pathological conditions. Most current blur assessment algorithms are designed and validated on consumer electronics such as television, video and mobile appliances. The few algorithms dedicated to medical images either requires a reference image or incorporate manual approach. For these reasons it is difficult to compare quality measures from different images and images with different contents. Furthermore, they will not be suitable in environments where large volumes of images are processed. In this report we propose a new blind blur assessment method for different types of MRI images and for different applications including automated environments. METHODS: Two copies of the test image are generated. Edge map is extracted by separately convolving each copy of the test image with two parallel difference of Gaussian filters. At the start of the multiscale representation, the initial output of the filters are equal. In subsequent scales of the multiscale representation, each filter is tuned to different operating parameters over the same fixed range of Gaussian scales. The filters are termed low and high energy filters based on their characteristics to successively attenuate and highlight edges over the range of multiscale representation. Quality score is predicted from the distance between the normalized mean of the edge maps at the final output of the filters. RESULTS: The proposed method was evaluated on cardiac and brain MRI images. Performance evaluation shows that the quality index has very good correlation with human perception and will be suitable for application in routine clinical practice and clinical research.

Advanced Image Enhancement Method for Distant Vessels and Structures in Capsule Endoscopy.

This paper proposes an advanced method for contrast enhancement of capsule endoscopic images, with the main objective to obtain sufficient information about the vessels and structures in more distant (or darker) parts of capsule endoscopic images. The proposed method (PM) combines two algorithms for the enhancement of darker and brighter areas of capsule endoscopic images, respectively. The half-unit weighted-bilinear algorithm (HWB) proposed in our previous work is used to enhance darker areas according to the darker map content of its HSV's component V. Enhancement of brighter areas is achieved thanks to the novel threshold weighted-bilinear algorithm (TWB) developed to avoid overexposure and enlargement of specular highlight spots while preserving the hue, in such areas. The TWB performs enhancement operations following a gradual increment of the brightness of the brighter map content of its HSV's component V. In other words, the TWB decreases its averaged weights as the intensity content of the component V increases. Extensive experimental demonstrations were conducted, and, based on evaluation of the reference and PM enhanced images, a gastroenterologist (Ø.H.) concluded that the PM enhanced images were the best ones based on the information about the vessels, contrast in the images, and the view or visibility of the structures in more distant parts of the capsule endoscopy images.

Bayesian framework inspired no-reference region-of-interest quality measure for brain MRI images.

We describe a postacquisition, attribute-based quality assessment method for brain magnetic resonance imaging (MRI) images. It is based on the application of Bayes theory to the relationship between entropy and image quality attributes. The entropy feature image of a slice is segmented into low- and high-entropy regions. For each entropy region, there are three separate observations of contrast, standard deviation, and sharpness quality attributes. A quality index for a quality attribute is the posterior probability of an entropy region given any corresponding region in a feature image where quality attribute is observed. Prior belief in each entropy region is determined from normalized total clique potential (TCP) energy of the slice. For TCP below the predefined threshold, the prior probability for a region is determined by deviation of its percentage composition in the slice from a standard normal distribution built from 250 MRI volume data provided by Alzheimer's Disease Neuroimaging Initiative. For TCP above the threshold, the prior is computed using a mathematical model that describes the TCP-noise level relationship in brain MRI images. Our proposed method assesses the image quality of each entropy region and the global image. Experimental results demonstrate good correlation with subjective opinions of radiologists for different types and levels of quality distortions.